Tractable Strong Outlier Identification 



FabriziO Angiulli, DEIS, University of Calabria 

Rachel Ben-Eliyahu-Zohary, Software Engineering Dept., Jerusalem College of Engineering 
Luigi Palopoli, DEIS, University of Calabria 



<: 



In knowledge bases expressed in default logic, outliers are sets of literals, or observations, that feature un- 
expected properties. This paper introduces the notion oi strong outliers and studies the complexity problems 
related to outlier recognition in the fragment of acyclic normal unary theories and the related one of mixed 
unary theories. We show that recognizing strong outliers in acyclic normal unary theories can be done in 
polynomial time. Moreover, we show that this result is sharp, since switching to either general outliers, 
cyclic theories or acyclic mixed unary theories makes the problem intractable. This is the only fragment of 
default theories known so far for which the general outlier recognition problem is tractable. Based on these 
results, we have designed a polynomial time algorithm for enumerating all strong outliers of bounded size 
in an acyclic normal unary default theory. These tractabUity results rely on the Incremental Lemma, an 
interesting result on its own, that provides conditions under which a mixed unary default theory displays a 
monotonic reasoning behavior. 

^ 1. INTRODUCTION 

Detecting outliers is a premiere task in data mining. Although there is no universal 

definition of an outlier, it is usually referred to as an observation that appears to de- 
viate markedly from the other observations or to be inconsistent with the remainder 
of the data [Hawkins 1980]. Applications of outlier detection include fraud detection, 
intrusion detection, activity and network monitoring, detecting novelties in various 
contexts, and many others [Hodge and Austin 2004; Chandola et al. 2009]. 
I Consider a rational agent acquiring information about the world stated in the form 

of a sets of facts. It is analogously relevant to recognize if some of these facts disagree 
^ with her own view of the world. Obviously such a view has to be encoded somehow 

using one of the several KR&R formalisms defined and studied in the Artificial Intelli- 
^ gence literature. In particular, for such a formalism to be suitable to attack interesting 

>^ KR problems it must be nonmonotonic so that it is possible to naturally exploit de- 

feasible reasoning schemas. Among the nonmonotonic knowledge representation for- 
malisms, Reiter's default logic [Reiter 1980] occupies a well-recognized role. 
0^ In a recent paper [Angiulli et al. 2008] formally defined the outlier detection prob- 

^ lem in the context of Reiter's default logic knowledge bases and studied some associ- 

ated computational problems. Following [Angiulli et al. 2008], this problem can be in- 
tuitively described as follows: outliers are sets of observations that demonstrate some 
properties contrasting with those that can be logically "justified" according to the given 
knowledge base. Thus, along with outliers, their witnesses, which are sets of observa- 
^ tions encoding the unexpected properties associated with outliers, are singled out. 

^ To illustrate, consider a case where during the same day, a credit card number is 

used several times to pay for services provided through the Internet. This sounds nor- 
mal enough, but add to that the interesting fact that the payment is done through 
different IPs, each of which is located in a different country! It might be the case that 
the credit card owner is traveling on this particular day, but if the different countries 
from which the credit card is used are located in different continents we might get 
really suspicious about who has put his hands on these credit card numbers. Another 
way to put it is to say that the fact that the credit card number is used in different con- 
tinents during the same day makes this credit card an outlier, and one of the probable 
explanations for such a phenomenon is that the credit card numbers have been stolen. 
This example is discussed further in the following. 
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As noted in [Angiulli et al. 2008], outlier detection problems are generally compu- 
tationally quite hard, with their associated complexities ranging from D^-complete to 
D3 -complete, depending on the specific form of problem one decides to deal with. For 
this reason, [Angiulli et al. 2010] singled out cases where a very basic outlier detec- 
tion problem, that is, the problem of recognizing an outlier set and its witness set, can 
be solved in polynomial time. A cumulative look at the results presented in [Angiulli 
et al. 2008; 2010], provides an idea of the tractability frontier associated with outlier 
detection problems in default logic. 

In this paper, we continue along this line of research and attempt to draw, as 
precisely as possible, such a tractability frontier. We want to depict the contour of 
a tractability region for outlier detection problems that refers to the well-known 
fragment of unary propositional default theories. In particular, motivated by the in- 
tractability of the general outlier recognition problem in all the classes of theories con- 
sidered thus far in the literature, we investigate this problem within further subsets 
of the classes already studied, such as the fragment of acyclic normal unary theories 
and the related one of mixed unary theories. We also introduce a new type of outliers 
which we will call strong outliers. 

Informally speaking, acyclic normal unary theories are normal unary theories char- 
acterized by a bounded degree of cyclicity, while strong outliers are outliers charac- 
terized by a stronger relationship with their witness set than in the general case. In 
this context, we have been able to prove that recognizing strong outliers under acyclic 
normal unary theories can be done in polynomial time and, moreover, that this result 
is sharp, since switching either to general outliers, to cyclic theories or to acyclic mixed 
unary theories makes the problem intractable. Notably, this is the only only fragment 
of default theories known so far for which the general outlier recognition problem is 
tractable. Based on these results, we designed a polynomial time algorithm for enu- 
merating all strong outliers of bounded size in an acyclic normal unary default theory. 

This algorithm can also be employed to enumerate all strong outliers of bounded size 
in a general normal mixed unary theory and, with some minor modifications, all the 
general outliers and witness pairs of bounded size. However, in this latter case, since 
the problems at hand are NP-hard, its worst case running time is exponential, even if 
from a practical point of view it can benefit from some structural optimizations which 
allows the algorithm to reduce the size of the search space. 

The rest of the paper is organized as follows. Section 2 recalls the definition of de- 
fault logics and that of the outlier detection task in the framework of default reasoning. 
Section 3 introduces the definitions of acyclic unary default theories, the definition of 
strong outlier, and provides a roadmap of the technical results that will be presented in 
the rest of the paper Section 4 presents intractability results while Section 5 presents 
some computational characterizations of mixed unary theories, the tractability result 
concerning strong outlier recognition that completes the layout of the tractability fron- 
tier, and the polynomial time strong outlier enumeration algorithm for acyclic unary 
default theories. To conclude. Section 6 runs through the complexity results presented 
in Sections 4 and 5 once more, this time focusing on their complementarity and com- 
menting also upon the application of the outlier enumeration algorithm within more 
general scenarios. The paper ends with our conclusions. 

2. OUTLIER DETECTION USING DEFAULT LOGIC 
2.1. Default Logic 

Default logic was introduced by Reiter [Reiter 1980]. We first recall basic facts about 
its propositional fragment. For T, a propositional theory, and S, a set of propositional 
formulae, T* denotes the logical closure of T, and -'S the set {-1(8)18 e S}. A set of 
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literals L is inconsistent if ^C. ^ L for some literal (, e L. Given a literal £, letter{£) 
denotes the letter in the literal £. Given a set of literals L, letter{L) denotes the set 
{A\A = letter{£) for some £ e L}^. 

2.1.1. Syntax. A propositional default theory A is a pair {D, W) where W is a set of 
propositional formulae and D is a set of default rules. We assume that both sets D and 
W are finite. A default rule 5 is 

g : ■ ■ ■ 

7 

where a (called prerequisite), ft, 1 < ? < m (called justifications) and 7 (called 
consequent) are propositional formulae. For 5 a default rule, pre{6), just{S), and 
concl{S) denote the prerequisite, justification, and consequent of S, respectively. 
Analogously, given a set of default rules, D = {Si, . . . ,Sn}, pre{D), just(D), and 
cond{D) denote, respectively, the sets {pre{5i), . . ., pre((5„)}, {just{5i), . . . ,just{dn)}, and 
{concl{Si), . .. , concl{Sn)}. The prerequisite may be missing, whereas the justification 
and the consequent are required (an empty justification denotes the presence of the 
identically true literal true specified therein). 

The informal meaning of a default rule 6 is as follows: If pre{5) is known to hold and 
if it is consistent to assume just{S), then infer concl{S). 

Next, we introduce some well-known subsets of propositional default theories rele- 
vant to our purposes. 

Normal theories. If the conclusion of a default rule is identical to the justification 
the rule is called normal. A default theory containing only normal default rules is 
called normal. 

Disjunction-free theories. A default theory A = {D, W) is disjunction free (DF for 
short) [Kautz and Selman 1991], if is a set of literals, and, for each S in D, pre{6), 
just{5), and concl{5) are conjunctions of literals. 

Normal mixed unary theories. A DF default theory is normal mixed unary (NMU 
for short) if its set of defaults contains only rules of the form where a is either 
empty or a single literal and /3 is a single literal. 

Normal and dual normal unary theories. An NMU default theory is normal 
unary (NU for short) if the prerequisite of each default is either empty or positive. An 
NMU default theory is dual normal (DNU for short) unary if the prerequisite of each 
default is either empty or negative. 

Figure 1 highlights the set-subset relationships between the above fragments of de- 
fault logic (acyclic theories to be defined in Section 3.2.2). 

2.1.2. Semantics. The formal semantics of a default theory A is defined in terms of 
extensions. A set £ is an extension for a theory A = {D, W) if it satisfies the following 
set of equations: 

— Eo = W, 

-for i > 0, Ei+i = E*u{^\ '^^^1^ eD,ae Ei,^l3i -/3„ s}, 

CXD 

-£=\jEi. 

i=0 

Given a default S and an extension £, we say that 6 is applicable in £ if pre{S) e £ 
and (^c e just{5)){-^c e £). 



As usual, for any letter a, we assume -■-■o = a. 
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Fig. 1 . A map of the investigated fragments default theory 

It is well known that an extension £ of a finite prepositional default theory A = 
{D, W) can be finitely characterized through the set Dg of the generating defaults for £ 
w.r.t. A (the reader is referred to [Reiter 1980; Zhang and Marek 1990] for definitions). 

Next we introduce a characterization of an extension of a finite DF propositional 
theory which is based on a lemma from [Kautz and Selman 1991]. 

Lemma 2.1. Let A = (_D, W) be a DF default theory; then £ is an extension of A if 
there exists a sequence of defaults Si, 6n from D and a sequence of sets Eq, Ei, En, 
such that for all i > 0; 

— Eo = W, 

— Ei = Ei^i U concl{5i), 

— pre{5,) C 

— {^c(^just{5i)){-^ce En), 

— (^(5 e D){pre{S) C En A concl{5) %EnA{^c& just{6)){^c € En)), 

— £ is the logical closure of En, 

where En is called the signature set of £ and is denoted liter{£) and the sequence of 
rules 6i,...,6nis the set Ds of generating defaults of£. 

Although default theories are non-monotonic, normal default theories satisfy the 
property of semi-monotonicity (see Theorem 3.2 of [Reiter 1980]). Semi-monotonicity 
in default logic means the following: Let A = (D, W) and A' = (£)', W) be two default 
theories such that D C D'; then for every extension f of A there is an extension £' of 
A' such that £ C£'. 

A default theory may not have any extensions (an example is the theory {{i^},^). 
Then, a default theory is called coherent if it has at least one extension, and incoherent 
otherwise. Normal default theories are always coherent. A coherent default theory 
A = {D, W) is called inconsistent if it has just one extension which is inconsistent. By 
Theorem 2.2 of [Reiter 1980], the theory A is inconsistent iff W is inconsistent. The 
theories examined in this paper are always coherent and consistent, since only normal 
default theories {D, W) with W a consistent set of literals are taken into account. 

The entailment problem for default theories is as follows: Given a default theory A 
and a propositional formula (j), does every extension of A contain cj)? In the affirmative 
case, we write A \= cj). For a set of propositional formulas S, we analogously write 
A ^ 5 to denote (V<^ e S')(A ^ ^). 
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2.2. Outliers in Default Logic 

The issue of outlier detection in default theories is extensively discussed in [AngiuUi 
et al. 2008]. The formal definition of outlier there proposed is given as follows. For a 
given set W and a collection of sets Si,. . . , 5„, Ws^ s„ denotes the set W \ (^i U 52 U 

...USn). 

Definition 2.2 (Outlier and Outlier Witness Set). [Angiulli et al. 2008] Let A = 
(D, W) be a prepositional default theory and let L c be a set of literals. If there 
exists a non-empty subset S of Wl such that: 

(1) {D, Ws) \= and 

(2) (D, Ws,l) y= -5 

then L is an outlier set in A and S is an outlier witness set for L in A. 

The intuitive explanation of the different roles played by an outlier and its witness 
is as follows. Condition (i) of Definition 2.2 states that the outlier witness set S denotes 
something that does not agree with the knowledge encoded in the defaults. Indeed, by 
removing S from the theory at hand, we obtain -^S. In other words, if 5* had not been 
observed, then, according to the given defaults, we would have concluded the exact 
opposite. Moreover, condition (ii) of Definition 2.2 states that the outlier i is a set of 
literals that, when removed from the theory, makes such a disagreement disappear. 
Indeed, by removing both S and L from the theory, -iS* is no longer obtained. In other 
words, disagreement for S is a consequence of the presence of L in the theory. To sum- 
marize, the set 5* witnesses that the piece of knowledge denoted by L behaves, in a 
sense, exceptionally, thus telling us that L is an outlier set and S is its associated 
outlier witness set. 

The intuition here is better illustrated by referring to the example on stolen credit 
card numbers given in the Introduction. A default theory A = (D, W) that encodes 
such an episode might be as follows: 

T-j J CreditNumber:-^MultipleI Ps 1 

^ ~ \ ^MultiplelPs J ' 

-W = {CreditNumber, MultiplelPs}. 

where the intuitive meaning of the literals is as follows: (a) CreditNumber is true if 
a given credit card number has been used for purchasing something today; (b) Mul- 
tiplelPs is true if that credit card number has been used from (computers located in) 
different continents today. Here, the credit card number might be stolen, for otherwise 
it would not have been used over different continents during the same day. Accordingly, 
L = {CreditNumber} is an outlier set here, and S = {MultiplelPs} is the associated 
witness set. This agrees with our intuition that an outlier is, in some sense, abnormal 
and that the corresponding witness testifies to it. 

Note that sets of outliers and their corresponding witness sets are selected among 
those explicitly embodied in the given knowledge base. Hence, we look at outlier detec- 
tion using default reasoning essentially as a knowledge discovery technique, whereby 
abnormalities and their presumable associated explanations are automatically singled 
out in the knowledge base at hand. For instance, it can be useful, to give one further 
example, when applied to information systems for crime prevention and homeland se- 
curity, because the outlier detection technique can be exploited in order to highlight 
suspicious individuals and/or events. Several examples for the usefulness of this ap- 
proach are given in [Angiulli et al. 2008] and in Section 3.2.1 below. 
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Table I. Complexity results for outlier detection (*=reported in [Angiulli et al. 2008], **=reported 
in [Angiulli et al. 2010]) 



Problem 


Outlier 
Type 


General 
Default 


DF 
Default 


(D)NU 
Default 


Acyclic 
(D)NU 
Default 


Outlier Recognition 


General 


S|'-c 
Th.4.3* 


Th.4.3* 


NP-c 
Th.3.6** 


NP-c 
Th.4.2 


Strong 


NP-hard 
Th.4.3 


NP-c 
Th.4.3 


P 

Th.5.7 


Outlier-Witness 
Recognition 


General 


D5-C 
Th.4.6* 


Th.4.6* 


P 

Th.3.1** 


P 

Th.3.1** 


Strong 


NP-hard 
Th.3.4 


P 

Th.3.3 


P 

Th.3.3 



3. CHARTING THE TRACTABILITY FRONTIER OF OUTLIER DETECTION 

This section introduces the main results that will be presented throughout the paper. 
Specifically, Subsection 3.1 recalls two main outlier detection tasks and the related 

complexity results linown so far; Subsection 3.2 introduces acyclic theories and a re- 
striction of the above defined concept of outlier, that we call strong outliers, and Subsec- 
tion 3.3 presents the plan of a set of results that will allow us to chart the tractability 
frontier in the context of prepositional normal mixed unary default theories. 

3.1. Outlier Detection Problems 

The computational complexity of discovering outliers in default theories under vari- 
ous classes of default logics has been previously investigated in [Angiulli et al. 2008]. 
In particular, the two main recognition tasks in outlier detection are the Outlier 
Recognition and the Outlier-Witness Recognition problems (also called Outlier{L) and 
Outlier {S){L), respectively, in [Angiulli et al. 2008]), and are defined as follows: 

- Outlier Recognition Problem: Given a default theory A = {D,W) and a set of 
literals L C W, is L an outlier set in A? 

- Outlier- Witness Recognition Problem: Given a default theory A = {D, W) and 
two sets of literals L cW and S C Wl, is L an outlier set with witness set S in A? 

Table I summarizes previous complexity results, together with the results that consti- 
tute the contributions of the present work that will be detailed later in this section. 

In particular, the complexity of the Outlier Recognition and the Outlier-Witness 
Recognition problems has been studied in [Angiulli et al. 2008] for general and 
disjunction-free (DF) default theories and in [Angiulli et al. 2010] for normal unary 
(NU) and dual normal unary (DNU) default theories. The results there pointed out 
that the general problem of recognizing an outlier set is always intractable (see The- 
orem 4.3 in [Angiulli et al. 2008] and Theorem 3.6 in [Angiulh et al. 2010]). As for 
recognizing an outlier together with its witness, this problem is intractable for general 
and disjunction-free default theories (see Theorem 4.6 in [Angiulli et al. 2008]), but 
can be solved in polynomial time if either NU or DNU default theories are considered. 
Regarding the latter result, it is interesting to note that, while for both NU and DNU 
default theories the entailment of a literal can be decided in polynomial time, deciding 
the entailment in DF default theories is intractable. 

Motivated by the intractability of the general Outlier Recognition problem on all 
classes of default theories considered so far, in this paper we take some further steps 
in analyzing the complexity of outlier detection problems in default logics in order 
to try to chart the associated tractability frontier To this end, in the next sections 
we consider further subsets of the classes already mentioned, referred to as Acyclic 
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Normal Unary anA Acyclic Dual Normal Unary theories, and a specific kind of outlier, 
which we will call Strong Outliers. Loosely speaking, the latter are characterized by 
a stronger relationship with their witness set than in the general case. The following 
Subsection 3.3 serves as an overview of main results of our complexity analysis. 

3.2. Strong Outliers and Acyclic Theories 

Next, the definitions of strong outlier set (Section 3.2.1) and of acyclic default theory 
(Section 3.2.2) are given. 

3.2.1. Strong Outliers. Recall the definition of outlier set already provided in Section 
2.2 (see Definition 2.2). 

Conditions 1 and 2 of the Definition 2.2 of outlier of Subsection 2.2 can be rephrased 
as follows: 

(1) {yieS){D,Ws) h-^, and 

(2) {3ieS){D,Ws,L)^^t 

In other words, condition 1 states that the negation of every literal £ e S must be en- 
tailed by {D, Ws) while, according to condition 2, it is sufficient for just one literal £ G S 
to exist whose negation is not entailed by {D, Ws,l)- Hence, there is a sort of "asym- 
metry" between the two conditions, which is the direct consequence of the semantics 
of the entailment established for sets of literals. 

It is clear that, at least from a purely syntactic point of view, the relationship be- 
tween the outlier set and its witness set can be strengthened by replacing the existen- 
tial quantifier in Condition 2 with the universal one, thus breaking the aforementioned 
asymmetry between the two conditions and obtaining the following definition of strong 
outlier set. 

Definition 3.1 (Strong Outlier). Let A = {D,W) be a prepositional default theory 
and let L c be a set of literals. If there exists a non-empty subset S o{Wl such that: 

(1) {WieS)iD,Ws) |=-£, and 

(2) {WeS){D,Ws,L)^^i 

then L is a strong outlier set in A and S is a strong outlier witness set for L in A. 

The following proposition is immediately proved: 

Proposition 3.2. If Lis a strong outlier set then L is an outlier set. 

Note that, in general the vice versa of Proposition 3.2 does not hold. 

In this paper we study the computational complexity of outlier detection problems 

when dealing with strong outliers. It turns out that, in general, it is easier to answer 
queries related to strong outliers. However, this is not the only reason we think that 
studying strong outliers is useful. We believe that using strong outlier detection can, 
in several cases, lead to a better representation of knowledge. To illustrate this point, 
consider a default theory A = {D,W) where D is the set of defaults {^^, ^|^, 

}, and W is the set of observations {ai,a2, a„, -161, If we use simple 

outlier detection, we get that for any possible nonempty subset L of {ai.0,2. .... a„} and 
any nonempty subset S of {-'bi,->b2, -'bn} that includes at least one literal ^foj with 
tti e L, S is a witness for L being an outlier. So, even if a default has "nothing to 
do" with a default for j ^ k (for example, one is about birds, the other is about 
students), both ^6^,} are a witness set for the outlier set {oj}. With strong outlier 
detection, we get a much better correspondence between the outliers and their witness 
sets. The set {-'bj,^bk} is not a strong witness for {aj}, but {-^bj} is. The following 
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concrete example that is an extension of the credit card scenario presented in the 
Introduction, demonstrates the above claim further. 

Recall that a credit card number is suspected to be stolen if it was used from sev- 
eral IPs in different continents during the same day. This example is extended next 
by adding a knowledge component related to violating normal behavioral patterns in 
using a cellular phone. Normally, almost all the numbers that people call are from 
their contacts list. In addition, for each cell phone user, there are hours of the day dur- 
ing which she normally does not use the phone. For example, most users would not 
use the phone during the night hours. Finally, for a typical cellphone user, there is a 
list of locations from which she normally calls. The knowledge described above can be 
summarized using the following defaults: 

(1) ^''^'^^^^^MMpMpf''^^^''' - Normally, credit card numbers are not used in different 
continents during the same day; 

(2) CeiiUseMfc _ (jyjfQ g^ands for "mostly from contacts") - Normally numbers dialed 
from a cell phone are mostly from the contact list; 

(3) ^""^Qu^Snir""' - Normally people do not use the phone during their "quiet time" 
- e.g. late at night; 

(4) Ce" ^^ewLoma°o°n ~ Normally cell phones are used in locations in which the device 
was used in the past. 

Now, suppose that a pickpocket stole Michelle's cellphone and purse from her hand- 
bag. She came home late and did not notice the theft till morning. While she was sleep- 
ing, the pickpocket could broadcast her credit card numbers through malicious servers 
over the Internet and use her cellphone to make expensive phone calls. A sophisticated 
crime prevention information system could automatically notice exceptional behaviors, 
and make the following observations: 

QuietTime. - calls are made from the device during abnormal hours; 

-iMfC. - It is not the case that most of the calls' destinations are from the phone's 

contact list; 

NewLocation. - The device is in a location where it has not been before; 
MultiplelPs. - The credit card number is used in different continents during the 
last day. 

Let us now consider the default theory A = {D, W), where D is the set of De- 
faults 1-4 introduced above, and W = {CreditNumber, CellUse, -^MfC, QuietTime, 
NewLocation, MultiplelPs}. According to the definition of outlier given in [Angiulli 
et al. 2008] (see Definition 2.2), we get that L = {CreditNumber} is an outlier and 
S = {^MfC, NewLocation, QuietTime, MultiplelPs} is a possible witness set for 
L. This last witness set is also a witness set for the outlier {CellUse}. However, al- 
though the observations MfC and QuietTime are in the witness set of the outlier 
{CreditNumber}, they do not appear really related to explaining why {CreditNumber} 
is an outlier. A similar consideration applies for the observation MultiplelPs, which is 
in the witness set of the outlier {CellUse}. 

One might suggest that, in order to improve the behavior demonstrated above, we 
should look for a minimal witness set. However, it is not always the case that minimal 
outliers are informative enough, since we would often like to look at rather complete 
sets of observations that support a suspicion identified by an outlier. In the example 
above, for instance, {-'MfC} is a minimal witness set for the outlier {CellUse}, but 
its superset {^MfC, NewLocation, QuietTime}, which is also a witness set for the 
outlier {CellUse}, provides more complete and compelling information. On the other 
hand, with the standard "weak" notion of outlier, however, non-minimal witnesses may 
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include irrelevant literals, as in the case, illustrated above, of the literal MfC which is 
in a witness set for the outlier {CellUse}. 

The notion of strong outlier presented above seems to adequately capture, in such 
scenarios as that depicted in this example situation, the notion of outlier and its 
"right" witness set. Indeed, if we use the definition of strong outlier we get that 
S = {^MfC, NewLocation, QuietTime, MultiplelPs} is neither a witness set for the 
outlier {CreditNumber} nor a witness set for the outlier {CellUse}. A witness set for 
the outlier {CellUse} is, instead, the set {-'MfC, NewLocation, QuietTime} or any of 
its nonempty subsets, while a witness set for the outlier {CreditNumber} is the set 
{MultiplelPs}. 

Next, we turn to the complexity issues. In order to mark the tractability landscape 
of the new strong outlier detection problem, we preliminarily provide two results, the 
former one regarding the tractability of the outlier-witness recognition problem and 
the latter one pertaining to its intractability. 

Theorem 3.3. Strong Outlier-Witness Recognition on propositional NU default 
theories is in P. 

Proof. The proof is immediate since the statement follows from the definition of 
strong outlier set (Definition 3.1) and the fact that the entailment problem on proposi- 
tional NU default theories is polynomial time solvable (as proved in [Kautz and Selman 
1991; Zohary 2002]). □ 

As for the complexity of the Strong Outlier-Witness Recognition problem on proposi- 
tional DF and general default theories, the following statement holds. 

Theorem 3.4. Strong Outlier-Witness Recognition on propositional DF default 
theories is NP-hard. 

Proof. The statement follows from the reduction employed in Theorem 4.6 of 
[AngiuUi et al. 2008], where it is proved that given two DF default theories Ai — {Di , 0) 
and A2 = (£>2,0), and two letters si and S2, the problem q of deciding whether 
((Al 1= si) A (A2 \= S2)) is valid can be reduced to the outlier-witness problem; that is, 
to the problem of deciding whether L = {32} is an outlier having witness set S = {^si} 
in the theory A{q), where A{q) = {D{q),W{q)) is the propositional DF default theory 
with D{q) = {£1^ | e Di} U £)2 and W{q) = {-si, S2}. Since the former problem 
is NP-hard, it follows from the reduction that the latter problem is NP-hard as well. 

In order to complete the proof, we note that a singleton witness set is always a strong 
witness set and, hence, the above reduction immediately applies to strong outliers as 
well. □ 

3.2.2. Acyclic NU and DNU theories. In this section, acyclic normal mixed unary default 
theories are defined. We begin by introducing the notions of atomic dependency graph 
and that of tightness of a NMU default theory. 

Definition 3.5 (Atomic Dependency Graph). Let A = (D, W) be a NMU default the- 
ory. The atomic dependency graph (V, E) of A is a directed graph such that 

-V = {I \ I is, a letter occurring in A}, and 

- E = {{x,y) I letters x and y occur respectively in the prerequisite and the consequent 
of a default in D}. 

Definition 3.6 (A set influences a literal). Let A = {D, W) be an NMU default the- 
ory. We say that a set of literals S influences a literal I in A if for some t & S there is a 
path from letter{t) to letter{l) in the atomic dependency graph of A. 
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Definition 3.7 {Tightness of an NMU theory ). 

The tightness c of an NMU default theory is the size c (in terms of number of atoms) 
of the largest strongly connected component (SCC) of its atomic dependency graph. 

Intuitively, an acyclic NMU default theory is a theory whose degree of cyclicity is fixed, 
where its degree of cyclicity is measured by means of its tightness, as formalized in the 

following definition. 

Definition 3.8 {Acyclic NMU theory). Given a fixed positive integer c, a NMU de- 
fault theory is said to be (c-)acyclic, if its tightness is not greater than c. 

We refer again the reader to Figure 1 in Section 2, that highlights the containment 
relationship among DF, NMU, NU, DNU, and acyclic default theories. 

For the sake of simplicity, in the following sections we refer to c-acyclic theories 
simply as acyclic theories. 

3.3. Main Results 

In this section, we provide a road-map and a brief description of the main results we 
are going to formally present and prove in Sections 4 and 5. 

It is clear from the definition of outlier that tractable subsets for outlier detection 
problems necessarily have to be singled out by considering theories for which the en- 
tailment operator is tractable. Thus, with the aim of identifying tractable fragments for 
the outlier recognition problem, we have investigated its complexity on acyclic (dual) 
normal unary default theories. These theories form a strict subset of normal unary de- 
fault theories already considered in [Angiulli et al. 2010] (other than being a subset of 
acyclic NMU theories), for which the entailment problem is indeed polynomially time 
solvable ([Kautz and Selman 1991; Zohary 2002]) and for which the outlier recognition 
problem is known to be NP-complete (Th. 3.6 of [Angiulli et al. 2010]). 

Unexpectedly, it turns out that recognizing general outliers is intractable even on 
such a rather restricted class of default theories as acyclic (dual) normal unary ones, 
as accounted for in the theorem whose statement is reported below. 

Theorem 4.2. Outlier Recognition for NU acyclic default theories is NP-complete. 

Note that the results for NU (DNU, resp.) theories immediately apply to E)NU (NU, 
resp.) theories, since given an NU (DNU, resp.) theory A, the dual theory A of A is 
obtained from A by replacing each literal £ in A with ->£ is a DNU (NU, resp.) theory 
that has the same properties of its dual. 

Unfortunately, the theorem reported above confirms that detecting outliers, even in 
default theories as structurally simple as acyclic NU and DNU ones, remains inher- 
ently intractable. Therefore, in order to chart the tractability frontier for this problem, 
we looked into the case of strong outliers. To characterize the complexity of this prob- 
lem, a technical lemma, called the incremental lemma, is needed. The Incremental 
Lemma provides an interesting monotonicity characterization in NMU theories which 
is valuable on its own. The statement of the incremental lemma is reported next. 

Lemma 5.5 (The Incremental Lemma). Let {D, W) be an NMU default theory, q a 
literal and S a set of literals such that W VJ S is consistent and S does not influence q 
in {D, W). Then the following hold: 

Monotonicity of brave reasoning:. If q is in some extension of (£>, W) then q is in 
some extension {D,W \J S). 

Monotonicity of skeptical reasoning:. If q is in every extension of {D, W) then q is 
in every extension of {D, W U S). 



10 



This lemma helps us to state an upper bound on the size of any minimal outlier witness 
set in an acyclic NMU (and, hence, also NU and DNU) default theory 

Lemma 5.6. Let {D, W) be a consistent NMU default theory and let L be a set of 
literals in W. If 5 is a minimal strong outlier witness set for L in {D, W), then letter{S) 
is a subset of a SCC in the atomic dependency graph of {D, W). 

Taken together, the following tractability result can be proved. 

Theorem 5.7. Strong Outlier Recognition for NU acyclic default theories is in P. 

The proof is informally as follows. Since by Lemma 5.6 the size of a minimal strong 
outlier witness set is upper bounded by ( , where c is the tightness of the theory, then 
the number of potential witness sets is polynomially bounded in the size of the theory. 
Moreover, checking conditions of Definition 2.2 can be done in polynomial time on NU 
theories, as the associated entailment is tractable. Based on these properties, a poly- 
nomial time procedure can be built that enumerates all the potential witness sets S 
for the outlier L and checks that S is actually a witness set for L. 

The formal proofs of Theorem 5.7 and of Lemmas 5.5 and 5.6 are reported in Section 
5. It is important to note that, actually. Lemma 5.6 cannot be exploited to prove the 
tractability of the Strong Outlier Recognition for NMU theories since, for these the- 
ories, the entailment problem remains intractable. Indeed, Theorem 4.5, reported in 
Section 4, shows that deciding the entailment is co-NP-complete even for NMU theo- 
ries with tightness equal to one. 

This latter result is completed by the following one. 

Theorem 4.3. Strong Outlier Recognition for NU cyclic default theories is NP- 
complete. 

In particular, both Theorems 4.2 and 4.3 make use of Lemma 4.1 (see Section 4 for 
the formal statement). Informally speaking, Lemma 4.1 establishes that, despite the 
difficulty to encode the conjunction of a set of literals using a NU theory, a CNF formula 

can nonetheless be evaluated by means of condition 1 of Definition 2.2 applied to an 
acyclic NU theory, provided that the size of S is polynomial in the number of conjuncts 
in the formula. 

The analysis accomplished in Section 5 (concerning tractability results) comple- 
ments that carried out in Section 4 (concerning intractability results), since Lemma 
5.6 establishes that the size of a minimal strong outlier witness set is upper bounded 
by the tightness of the NU theory. Hence, imposing both theory acyclicity and outlier 
strongness prevents the use of Lemma 4.1 for reducing the CNF satisfiability problem 
to the strong outlier recognition one. 

Summarizing, recognizing strong outliers under acyclic (dual) normal default theo- 
ries is the only outlier recognition problem known so far to be tractable. Furthermore, 
this result is indeed sharp, since switching either to general outliers, to cyclic theories, 
or to acyclic NMU theories makes the problem intractable. 

Based on the above results, we designed a polynomial time algorithm for enumerat- 
ing all strong outliers of bounded size in an acyclic (dual) normal unary default theory. 
The algorithm can also be employed to enumerate all strong outliers of bounded size 
in a general NMU theory and, with some minor modifications, all the general outliers 
and witness pairs of bounded size. However, in this latter case, since the problems at 
hand are NP-hard, its worst case running time will remain exponential, even if, from 
a practical point of view, it can benefit from some structural optimizations, based on 
Lemmas 5.5 and 5.6, which would allow to reduce the size of the search space. 
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All complexity results presented in the rest of this work, together with those already 
presented in the literature, are summarized in Table I, where the problems lying on 
the tractability frontier are underlined. 

The next two sections present technical proofs associated with the aforementioned 

results. 

4. INTRACTABILITY RESULTS 

In this section we present the intractability results, referred to in Section 3, concerning 
the NP-completeness of outlier recognition in acyclic NU default theories (Theorem 
4.2), the NP-completeness of strong outlier recognition in cyclic NU default theories 
(Theorem 4.3), and the co-NP-completeness of the entailment for acyclic NMU default 
theories (Theorem 4.5). 
First, some further definitions are needed. 

Let T be a truth assignment to the set xi, . . . ,a;„ of boolean variables. Then Lit{T) 
denotes the set of literals {£i, . . . ,£„}, such that £i is if T(x,;) = true and it is -iXj if 
T(.Tj) = false, for i = 1, . . . ,n. 

Let L be a consistent set of literals. Then 71 denotes the truth assignment to the set 
of letters (boolean variables) occurring in L such that, for each positive literal p <E L, 
Tl{p) = true, and for each negative literal -^p e L, Tl{p) = false. 

We start by providing a technical lemma (Lemma 4.1) which shows how the evalu- 
ation of the truth value of a CNF formula under a specific assignment can be accom- 
plished in a way that relates it to the definition of outliers. 

Lemma 4.1. For each boolean formula $ in 3CNF having m conjuncts, there exists 
a NU propositional default theory (D($), W($)) whose size is polinomially bounded in 

a set of literals S'("I>) C W{^), and a set of letters ci, . . . , c„j occurring in D{<^), such 
that $ is satisfiable if and only if{D{^), M^($)s(<i>)) |= ^(S'($) U {ci, . . . , Cm}). 

Proof. Let $ = f{X) be a boolean formula in 3CNF, where X = xi, . . . , a;„ is a set 
of variables, and f{X) = Ci A . . . A Cm, with Cj = tj^i V tj^2 V tj^s, and each tj,2, ij,3 
is a literal on the set X, for j = 1, . . . , m. Let A($) = (r)(<I>) be a NU default 

theory associated with where W{^) is the set {xi, . . . ,x„} of letters and is the 
set of defaults £>i U £>2 U D3, with: 

= {4S = ^^^^^^|i = l,...,m;fc = l,2,3}, 

D, = /<5f) = i^|i = l,...,n}, 

where ci, . . ., Cm and yi, ■ ■ ■, Un are new letters distinct from those occurring in and 
a{x.i) = Xi and (T{^x.i) = yi, for i = 1, . . . , n. Let i? be a subset of {xi, . . . , x„}. In the 
rest of the proof, (t{R) will denote the set {(t{x) \ x e R). Moreover, cr^^ will denote the 
inverse of a, that is to say (j~^{xi) = Xi and (T{yi) = -iXj, and (t~^{R) = {(j~^{x) \ x e R}. 

Next, it is shown that there exists a set of literals S{<^) c W{<^) such that 
(Z)(4>), W($)5(<i,)) h ^{S{^) U {ci, . . . , c„}) if and only if $ is satisfiable. 

First, the following claims are proved. 

Claim 1. Let S C W{<^). For each extension S of (-D($), it is the case that 

— for each Xi e S, -■Xi e £ and yi e £, and 
— for each Xi ^ S, Xi G £. 
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Proof of Claim 1: Indeed, any extension £ of {D{<^), W['^)s) is such that (i) for each 

Xi ^ S, Xi ^ £ and -^Xi e £, since .t^ ^ W{<^)s, no letter x, appears in the consequence 

of a default in -D($), and due to defaults 6 {I; (ii) for each Xi e S,yi e £, due to defaults 
(52^'; and (mO for each Xi ^ S, Xi G £, since Xj e W($)s. □ 

Claim 2. Lei 5 C W{^). Then, there exists an extension £* of (-D($), W{<^)s) such 
that 

— for each Xi e S, -'Xi e £* and yt e £*, and 
— for each Xi ^ S, Xi G £* and -'yi e £*. 

Proof of Claim 2: Note that for each Xi ^ S either ->yi e £ (due to defaults s[^j) or 

yi <E £ (due to defaults 62^^). The proof then follows from Claim 1. □ 
Now, the main proof of Lemma 4.1 can be resumed. 

(If part) Assume that there exists a subset S{^) C W{^) such that 

1= ~^iSi^) U {ci, . . . , Cm}). Note that the set S'(<f>) may contain some let- 
ters from the set {xi, . . . ,a;„}. Further, the defaults in the set £>i serve the purpose 
of introducing the letters yi in the current extension of the theory (D{^),W{^)s). In 
particular, the letter yi is intended to represent the negation of letter Xi and it is intro- 
duced since in NU theories negated literals cannot be specified as the prerequisite of a 
default. 

Moreover, the defaults in the set D2 evaluate the CNF formula $ by using the truth 
value assignment encoded by the letters and yt belonging to the current extension 
of the theory {D{^),W (<&)s). In particular, if the jth conjunct Cj of $ is true, then the 
literal -^cj belongs to the current extension. 

By Claim 2, there exists an exension £* of (-D(^), W{^)s) such that the set of literals 
R = a~^{£* n {X UY)) is consistent and precisely encodes a truth value assignment to 
the variables in the set X, namely Tr. 

To conclude, since (£>(<!>), VK($)s) |= ^ci A . . . A ^c^, by the defaults in the set D2, Tr 
is a truth value assignment to the variables in the set X that makes the formula f{X) 
true and, hence, $ is satisfiable. 

(Only If part) Assume now that the formula $ is satisfiable, and let Tx be a 
truth value assignment to the variables in the set X that makes f{X) true. Next 
it is shown that S{^) = {xi \ Tx{xi) = false} is such that (£)($), |= 
-(5($)U{ci,...,c„}). 

Let f be a generic extension of (-0(4"), W{^)s(^))- As for the letters Xi belonging to 

due to defaults 5^^], ^Xi belongs to £. 
It remains to show that -^Cj e £, for each j e {1, . . . , m}. By Claim 1, for each exten- 
sion £ of (£>(«>), VF($)s(<i>)), the set Q = n (X U F) is a superset of cr(Lit(Tx)). That 
it to say, it might be the case that both Xi and are in the set Q, for a letter Xi not in 

Since the defaults in the set D2 encode the boolean formula obtained from $ by 

substituting each negative literal with the positive literal yi, having both Xi and 
yi in Q corresponds to assuming that both the literals x^ and ^x, are to be considered, 
informally speaking, true within (f). 

Indeed, as already observed, $ is evaluated by means of a different formula, say 
obtained from $ by substituting each occurrence of the negated literal -iXj with the 
letter y, (see the defaults in 02). If the literals in Q form a superset of the literals 
associated with a truth value assignment making <& true, they also encode a satisfying 
truth assignment that makes the formula true. Thus, by virtue of the defaults in 
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Fig. 2, 



The atomic dependency graph of the theory A'($) in Theorem 4.2. 



D2, each extension £ of (D($), 1/F($)5($)) is such that -iCj belongs to £, for j = 1, . . . , m. 
This closes the proof. □ 

Theorem 4.2. Outlier Recognition for NU acyclic default theories is ^^-complete. 

Proof. (Membership) Membership in NP immediately follows from the analogous 
result of the Outlier Recognition problem for NU default theories (see Theorem 3.6 of 
[AngiuUi et al. 2010]). 

(Hardness) Let $ be a boolean formula in 3CNF (as described in Lemma 4.1). Check- 
ing 3CNF formulae satisfiability is well known to be NP-complete [Papadimitriou 
1994]. 

Recall the default theory {D{<b), W{^)) described in the proof of Lemma 4.1. In order 
to prove the NP-hardness of the problem at hand, the NU default theory A'($) = 
(£)'($), W'{<^)) is associated with where W'{^) is the set 



of letters, with / , cq , ci , . . . , c„i being new letters distinct from those occurring in and 

£>'($) is the set of defaults £»($) U i:»4 U U Dr, with: 



The theory A'($) can be built in polynomial time and, moreover, is consistent. 

Now we can observe that the theory A'($) has tightness equal to 1 — the atomic 
dependency graph of A'(<I>) is shown in Figure 2. Next, it is shown that L = {-^1} is an 
outlier in A'(<I>) if and only if $ is satisfiable. 

(Only If part) Assume that L = {-'1} is an outlier in A'($). Since L is an outlier, 
there exists an outlier witness set S C W'{^) \ L for L. 

Now we show that the set S contains all the literals in the set {cq, ci, . . ., Cm}- The 
only literal of \ L whose negation is not entailed by (!?'(<&), W'{*^)s,l) is cq, since 

this theory entails cq by the defaults in Dq. Thus, cq belongs to S, for otherwise {-•/} 



VF($)U{^Z,co,ci,..., 
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would not be an outlier set. Since (D'($), T4^'($)s) |= and co e S, by the defaults in 
I>5, S contains the set {cq, ci, . . . , Cm}, for otherwise (£)'($), |= -icq cannot be 

true. 

Let C be {ci, . . . ,Cm} and let i? be \ (C U {cq}). Next we show that there does 
not exist an extension £ of {D{^),W{^)r) such that £ ^ ^(i? U C) and, hence, that 

(D(a>),w-($);?,) h-(i?uc). 

First, note that W(*)s equals W{^)r U {^0- while C £)'($). Since, does 

not occur in any default of the theory {D{^),W{^)) and since -'I occurs in W'{^), 
then the semi-monotonic relationship still holds between theories (-D($), W{^)ji) and 

Assume that there exists an extension £ of {D{^),W{^)r) such that £ 2 U 
C). Then, there exists either a literal -iXj/ ^ £ {i' € {!,...,«}) or a literal -'Cji ^ 
f 0' e {l,...,rn}). Moreover, by semi-monotonicity there exists an extension £' of 
(£>'($), H^($)5) such that f I) £. However, since no literal {1 < i < n) and -^Cj 
(1 < j < m) occur in any default in the set D^U D^U D^, the missing literal {xi' or Cj') 
does not belong to £' as well and (£>'($), W'{^)s) ^ ^5", a contradiction. Thus, it can 
be concluded that W{^)r) \= -'{R U C) and, by Lemma 4.1, that the formula $ 

is satisfiable. 

(If part) Assume now that the formula $ is satisfiable. Then, by Lemma 4.1, there 
exists a set of literals R C {xi, . . . ,Xn} such that {D{<^),W{<^)r) |= ^{R U C). 

Let S be i?UCU{co}. We show next that (-D'($), VF'($)s) h -^S. By semi-monotonicity, 
it follows that there exists at least one extension £' of (!)'($), such that f 3 

U C). Moreover, since no literal occurring in the set of defaults D{^) also occurs in 
the consequence of any default in D4 U U Dq, each extension £' of (Z)'(<1>), 5) is 

such that £' 2 -^{R U C). Then, for each extension £' of (£>'($), there exists at 

least one letter Xi or pi in f . Indeed, if Xi ^ 5, then Xi G £'. Otherwise, yi, . . . , y„ e f . 
Thus, by virtue of the defaults in the set 1)4 the literal -iCq belongs to £'. It can be 
concluded that VK'($)s) h ^-S*- 

Consider now the theory {D'{^),W'{^)s,{^i})- By defaults in the set Dq, there 
exists at least one extension £' of this theory such that cq e f and, thus, 
(£)'($), W^'(<!>)5 {-,;}) ^ -■S'. In other words, L = {^1} is an outlier set with associated 
outlier witness set S in A'($). This closes the proof. □ 

Theorem 4.3. Strong Outlier Recognition for NU cyclic default theories is NP- 
complete. 

Proof. (Membership) Since strong outliers are a subset of general outliers, the 
result follows from the membership in NP of the Outlier Recognition problem for NU 
default theories (see Theorem 3.6 of [AngiuUi et al. 2010]). 

(Hardness) Let $ be a boolean formula in 3CNF (as described in Lemma 4.1); the 
satisfiability detection of <!> is well known to be NP-complete [Papadimitriou 1994]. 

Recall the default theory (Dii^), Tl (<I>)) described in the proof of Lemma 4.1. In order 
to prove the NP-hardness of the problem at hand, the NU default theory A'($) = 
(D'($), W'{^)) is associated with where W'{^) is the set 



of letters, with l,c\, . . . ,Cm being new letters distinct from those occurring in and 
£)'($) is the set of defaults U D4 U U Dq, with: 



T^(<I>)UH,ci,...,c„} 
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D, = |<5g = ^^,45 = :^|j = l,...,m|,and 

De = |^f^ = ^,4') = ^|j = l,...,m|. 

The theory A'($) can be built in polynomial time and, moreover, W($) is consistent. 
By the defaults in the set D4, the theory A'($) is cyclic. 

Next it is shown that L = {-'1} is a strong outlier in A'($) if and only if $ is satisfi- 
able. The following claim will be useful toward this end. 

Claim 3. Let R be a subset of W'{^). If there exists an extension £ of 
{D'(^),W'{^)r) such that the letter f belongs to £, then for each £ € R \ {-■/} 

Proof of Claim 3: Note that if the letter / can be entailed, then for each £ € R \ {-•/}, 
by defaults S^^^ and §2% there exists at least one extension of {A'{^),W'{^)r) such 
that £e£e and, consequently {A'{<P),W'{<^)r) ^ ^£. □ 
We can now resume the Theorem's proof 

(Only If part) Assume that L = {-•/} is a strong outlier in A'($). Since L is a strong 
outlier, there exists a strong outlier witness set S C T4^'($) \ L for L. 

By Claim 3, the letter / cannot belong to any extension of (£>'($), W'{^)s), for other- 
wise S is not a (strong) outlier witness set inasmuch as condition 1 of Definition 2.2 is 

not satisfied. Thus, by defaults 6[^j, it is the case that the set S contains all the literals 

in the set {ci, . . ., c™}. 

Let C be {ci, . . . , c™} and let i? be S* \ C. By using exactly the same argument em- 
ployed in the Only If part of Theorem 4.2, it can be proved that {D{^),W{^)r) \= 
-'{R U C), and, by Lemma 4.1, it can be concluded that the formula $ is satisfiable. 

(If part) Assume now that the formula $ is satisfiable. Then, by Lemma 4.1 there 
exists a set of literals i? c {.xi, . . .,Xn} such that {D{<^),W{<^)b) |= ^(i?U C). 

Let SheRUC.We show that iD'{^), \= -^S. By semi-monotonicity it follows 

that there exists at least one extension £' of {D'{^),W'{^)s) such that £' 3 u 
C). Moreover, since no default in D4 U D5 U Dq may belong to the set of generating 
defaults of (£>'($), VK'($)s), it is the case that each extension £' of T'F'($)s) is 

such that £' D ^(i? U C). Indeed, / cannot be entailed by this theory since no letter 
among c\,.. .,Cm appears in W'{^)s and in the consequence of any default in _D'($) 
and, moreover, -^l belongs to W'{<^)s- It can thus be concluded that {D'{<b), W'{^)s) \= 

Since by virtue of the defaults in the set D^, there exists at least one extension £ of 
the theory {D' ,W' {<^) s,{-^i}) such that / belongs to £, by Claim 3 it is the case that 
(£)'($), W {^)s,{-^i}) satisfies condition 2 of Definition 2.2 and, hence, that 5 is a strong 
outlier witness set for L. Thus, L = {-■/} is a strong outlier set in A'($). This closes the 
proof □ 

As far as the complexity of the Strong Outlier Recognition problem on propositional 
DF and general default theories is concerned, the following result directly follows from 
Theorem 4.3. 

Corollary 4.4. Strong Outlier Recognition for DF and general default theories is 
NP-hard. 

Before concluding this section, for the sake of completeness, the complexity of the 
entailment problem for propositional NMU default theories is analyzed. 
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Theorem 4.5. Let Abe a NMU propositional default theory and let I be a literal. 
Then the entailment problem A \= I is co-NP-complete. 

Proof. (Membership) Membership in co-NP follows immediately from membership 
in co-NP of the entailment problem for disjunction-free propositional default theories 
[Kautz and Selman 1991]. 

(Hardness) Let $ be a boolean formula in 3CNF on the set of variables X = 
xi, . . .,Xn, such that $ = Ci A . . . AC^, with Ck = V tfc,2 V tfc,3, and eachtk,i,tk,2,tk,3 
is a literal on the set X, for k = 1, . . . ,m. The default theory A($) = (D($), 0) is asso- 
ciated with where is DiU D2U D^, with: 

D2 = 1^^^^ |fc=l,...,m;j = l,2,3|, and 

( ■■ ^Ck ^Ck : I . \ 
-^3 = 1^^' — I — l^ = l'---'™p 

where / is a new letter distinct from those occurring in $. It is shown next that $ is 
unsatisfiable iff A($) |= /. We recall that the unsatisfiability problem of a 3CNF is a 
well-known co-NP-complete problem. 

Consider a generic extension £ of A(<l>). From the rules in the set Di, £ is such that 
for each i = 1, . . . , n, either Xi e E or -^Xi G E. 

(Only If part) Suppose that $ is unsatisfiable. Then, for each truth assignment T on 
the set of variables X, there exists at least a clause, say Cf^r), 1 < f{T) < m, that is 
not satisfied by T. Because of the rules in the set D2, Cf^Tsncxu^x)) ^ ^' from rules 
in the set D3, --Cf^^remxu^x^) G ^ and I e £. 

(If part) Suppose that A($) \= I. Then, for each extension £ of A(<E>), there exists 
!){£), 1 < 9i£) < "1, such that ^fg(£) G £■ For each truth assignment T on the set of 
variables X, let E{T) denote the set containing all the extensions £ of A($) such that 
£ D Lit{T). Then, for each £ e E{T), -^Cg{£) G £ implies that none of the rules in the set 
D2 having Cg(£) as their conclusion belong to the set of generating defaults off. Thus, 
the clause Cg(^s) is not satisfied by T. As this holds for each truth assignment T, $ is 
unsatisfiable. □ 

From the above result, the complexity of the entailment problem for propositional 
acyclic default theories can be straightly derived, as done in the following. 

Corollary 4.6. Let Abe a NMU propositional acyclic default theory and let I be a 
literal. Then the entailment problem A\= lis co-NP-complete. 

Proof. Note that the theory employed in the reduction of Theorem 4.5 has tight- 
ness 1. □ 

5. TRACTABILITY RESULTS 

In this section, we prove several tractability results about outlier detection in default 
reasoning which, together with the intractability ones given in the previous section, 
provide a sufficiently accurate illustration of the tractability border for the outlier 
recognition problem. We begin with discussing some new characterizations about com- 
puting with NMU theories (Section 5.1) and then provide our tractability results (Sec- 
tion 5.2). 
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5.1. Computational characterization of NlUlU tiieories 

This section is devoted to proving the Incremental Lemma, which is the basic result 
used to single out tractable cases of outlier detection problems. Alongside, the Incre- 
mental Lemma provides an interesting monotonicity characterization in NMU theories 
which is valuable on its own. 

Definition 5.1 (Proof). Let A = [D, W) be an NMU default theory, let I be a literal 
and E a set of literals. A proof of I w.r.t. A and E is either I by itself, if I G W, or r 
sequence of defaults di, Sn, such that the following holds: (1) I is the consequence of 
6n, (2) -^l ^ E, and (3) for each 5i, 1 < i < n, either 5i is prerequisite-free, or pre{Si) S W, 
or (5i, (5j_i is a proof of pre(5i) w.r.t. A and E. A proof is minimal if it is not possible 
to make it shorter by deleting a default from it. 

Note that every sequence of generating defaults for an extension E has a subsequence 
which is a proof for every literal in E. 

Lemma 5.2. [Ben-Eliyahu and Dechter 1994] Let A = [D, W) he an NMU default 
theory, let I be a literal and E an extension of A. Then I is in E iff there is a proof of I 
w.r.t. A and E. 

Definition?).^ [Satisfaction of an NMU default). A set of literals E satisfies an 
NMU default S = ^ iff at least one of the following three conditions hold: (1) y ^ E,or 

(2) -nx e E, or (3) xe E. 

Theorem 5.4. [Ben-Eliyahu and Dechter 1994] A set of literals E is an extension 
of an NMU consistent default theory (D, W) iff the following holds: (1) W C E, (2) E 
satisfies every default in D, and (3) every literal in E has a proof w.r.t (D, W) and E. 

The Incremental Lemma, reported below, characterizes a monotonic behavior of 
NMU theories. 

Lemma 5.5 (The Incremental Lemma). Let {D, W) be an NMU default theory, 
q a literal and S a set of literals such that W U S is consistent and S does not influence 
q in (D, W). Then the following hold: 

Monotonicity of brave reasoning:. If q is in some extension of {D,W) then q is in 
some extension of {D, W U S). 

Monotonicity of skeptical reasoning:. If q is in every extension of{D, W) then q is in 
every extension of (D, W U S). 

Proof. (Monotonicity of brave reasoning) Suppose that for some extension E' 

of (D, W), q e E', and let a = 5i, ...,5k be a proof of q wrt {D, W) and E' . Let E„ be an 
extension of A = (cr, U S). We will show that a is also a proof of q w.r.t. A and E^. 
Clearly each prefix of (t is a proof w.r.t [D, W) and E' . We prove that each prefix is also 
a proof w.r.t. A and E„. The proof is by induction on the size of the prefix. 

Case the prefix is of size 1.. In this case, the proof is either a literal that belongs 
to W, or a default of the form I : t/t, where I G W and t is a literal. If the proof 
is a literal that belongs to W, it obviously belongs also to W U S. Suppose the 
consequence of S is some literal t. It cannot be the case that a default having a 
consequence -^t belongs to a because ct is a proof and 5 is in a. It also cannot be the 
case that -'t €W because a is a proof w.r.t. (D, W). Last, it also cannot be the case 
that -'t G S because S does not influence q in (D, W) and 6 is part of the proof of g 
wrt. {D, W) and E'. Hence S is applicable in E^, and so t G E^. 
Case the prefix is of size greater than 1.. Suppose S is the last default in the prefix. 
If S has a prerequisite I, then by the induction hypothesis, I G E^. Suppose the 
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consequence of S is some literal t. By arguments similar to the induction base case, 

S is applicable in B^, and so t e E^. 

Since cr C £), by semi-monotonicity of normal default theories, there is an extension E 
of {D, W U S) such that E„ C E. Since q e E^, q € E. 

(Monotonicity of skeptical reasoning) Suppose that q is in every extension of 
(£>, W) and assume conversely that there is an extension E' of {D, W U S) such that 
q ^ E' . Let a = (5i,...,(5„ be a sequence of generating defaults of E' as defined in 
Lemma 2.1. a will be modified so that it will not have defaults with consequences that 
are influenced by S. This is done as follows: 

(1) Delete from a all rules of the form : l/l or t : l/l where I belongs to S. 

(2) For /i = 1 to n, if 6h was not deleted in the previous step and the prerequisite of Sh 
is not in W and not a consequence of any default which is before 5h in a and was 
not deleted, then delete Sh from a. 

Let as be the sequence of defaults left in a after the modification described above. 

Let Ea be an extension of (crs, W). 

Claim 4. For every default 6 in as, the consequence of 6 is in E„. 

Proof: By induction on the index of S in the sequence as- 

Case the index of 5 is 1.. If 5 has a prerequisite I, then by the way as was con- 
structed, / must belong to W. Suppose the consequence of 5 is some literal t. It 
cannot be the case that a default having a consequence -^t belongs to as because as 
is a subset of a set of generating defaults and 5 is in as- It also cannot be the case 
that -'t € W because as is a subset of the generating defaults of (D, W U S) and S 
is in as- Hence, 6 is applicable in E^, and sotGE^. 

Case the index of 6 is greater than 1.. By the induction hypothesis, if 6 has a pre- 
requisite then it belongs to E„. By arguments similar to the ones given in the 
induction base part, the consequence of ^ belongs to E^. 

By semi-monotonicity of normal default theories, there is an extension E of (D, W) 
such that Ea- C E. By the assumption that q belongs to every extension of {D, W), 
q G E. Then, by Lemma 5.2, there is a sequence of defaults tt = i5i, Sk such that tt is 
a proof of q w.r.t. {D, W) and E. Since q ^ E', for some I < j < k, Sj ^ as- Let i be the 
minimum index such that 6^ e n and Si ^ as- It must be the case that Si = I -.t/t where 
/ might be empty. We now consider two cases. 

Case Si is applicable in E'.. If this is the case. Si is one of the generating defaults 
of £", or in other words. Si G a. Since Si ^ as, and Si G a, it must be the case that 
S influences t in {D, W). But Si is a default in a proof of q w.r.t. (£>, W) and E, so S 
influences q in {D, W), a contradiction. 

Case Si is not applicable in E'.. Since i is the minimal index such that Si ^ as, 
it must be the case that -^t belongs to E' (otherwise Si would have been in the 
generating defaults of £", and since S does not influence t in D, Si would belong to 
as). Since Si = I : t/t is part of a proof of q w.r.t. (D, W) and E, it must be the case 
that -^t ^ W. Clearly, since S does not influence t in {D, W), -^t is not in S either. 
But Si is not applicable in E', which is an extension of {D, WUS), so there must be a 
rule S <E D such that S is applicable in E' and the consequence of S is ^t. So S is one 
of the generating defaults of E'. Since 5* does not influence tin D, S must belong to 
as- By Claim 4 above, -'t belongs to E^r, and by semi-monotonicity, -^t belongs to E. 
So Si = I : t/t cannot be part of a proof of g w.r.t. {D, W) and E, a contradiction. 

□ 
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5.2. Tractability results for outlier detection 

This section presents a characterization of minimal strong outher witness sets in 
prepositional NMU theories which is then exploited in order to provide a polynomial 
time algorithm for enumerating all strong outlier sets of bounded size in a NU prepo- 
sitional acyclic default theory 

Lemma 5.6. Let {D, W) be a consistent NMU default theory and let L be a set of 
literals in W. Then, if S is a minimal strong outlier witness set for L in {D, W), letter {S) 
is a subset of a SCC in the atomic dependency graph of {D, W). 

Proof. Let L be a set of literals in W and let S be an outlier witness set for L in 
(D, W). By definition, the following must be true: 

(1) {W G S){D, Ws) \= and 

(2) {\f£ e S){D,Ws,l) ^ ^i. 

We can partition S into disjoint sets Si,...,Sn such that the following holds: 

(1) [JtlS^ - S. 

(2) For each 1 < i < n, it I and q are in S then I G Si and q £ Si if and only if letter{l) 
and letter (q) are in the same SCC in the atomic dependency graph of (D, W). 

(3) For each q, l € S and for each 1 < i < n — 1, if I € S^ and q € Sj for some 2 < 
j < n such that i < j, then there is no path in the atomic dependency graph from 
letter{q) to letter{l) (that is, the Si's are ordered according to the the reachability 
relationship induced on the the atomic dependency graph). 

Next we show that the following holds: 

(1) (V£g 5i)(£>,M^Sx) N-^.and 

(2) {\JI€S^){D,Ws,,l)V=^^- 

That means that Si is a strong outlier witness set for L. Since is a subset of an SCC 
in the atomic dependency graph of (D, W) this will complete the proof 

We first consider the condition (V£ G VKgJ \= -^L It is given that (V^ G 

S)(p,Ws) h In another notation: (V^ G S){D,Ws^,...,Sr) \= so that clearly 
(V£ G S{){p^Ws^,...,s^ 1= Since there is no path in the atomic dependency graph 
from any letter of a literal in . . . , S'n to a letter of a literal in S\, we can use the 
Incremental Lemma to conclude that (V£ G S\)(p, Ws^) h 

Now consider the condition (V^ G Sx)(B,Ws^ l) ^ Since (V£ G S)(D,Ws l) ^ 
clearly (V^ g Si){D,Ws,l) W It cannot be the case that 3£ G such that 
{D, Ws,l) ^ ~'£ but {D, Ws,L U {S2, <S'„}) 1= -^£, since there is no path in the atomic 
dependency graph from 82, ...,Snto Si. Hence (V£ G ^i), {D, Ws,l U {^2, ^ -.^ 
or, in an equivalent notation, (V^ e Si){D, Wsi,l) ^ ~'£- □ 

Theorem 5.7. Strong Outlier Recognition for NU acyclic default theories is in P. 

Proof. Given a NU default theory {D, W) of tightness c and a set of literals L from 

W, by Lemma 5.6 a minimal outlier witness set S for L in {D, W) has a size of at most 
c, where c is the maximum size of an SCC in the atomic dependency graph of {D, W). 

Thus, the strong outlier recognition problem can be decided by solving the strong 
outlier- witness recognition problem for each subset S of literals in Wl having a size of 
at most c. Since the latter problem is polynomial time solvable (by Theorem 3.3) and 
since the number of times it has to be evaluated, that is 0(|Ty|'^), is polynomially in 
the size of the input, then the depicted procedure solves the strong outlier recognition 
problem in polynomial time. □ 
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1 


Input: A = {D,W)-a NU default theory. 


2 


Output: Out - the set of all strong outlier sets L in A s.t. \L\ < k. 


3 


let Ci, . . . , Cjv the ordered oCCs in the atomic dependency graph oi A; 


4 


set (Jut to It); 


5 


tor z = i..iV do 


6 


tor ail a G W s.t. letter{b ) )= Ci do 


7 


if fV^ € S)(D Wv) \= -n£ then 


8 


for all L C T^s s.t. \L\ < k and letter{S) C (Ci U . . . U C^) do 


9 


if (V£ e £')(£', ^ then 


10 


: set Out to Out \j{L}; 


11 


end if 


12 


end for 


13 


end if 


14 


end for 


15 


: end for 



Fig. 3. Algorithm Outlier Enumeration. 

To take a step further and present an outlier enumeration algorithm, a known propo- 
sition is recalled. 

Proposition 5.8. (proved in [Kautz and Selman 1991; Zohary 2002]) Let A be 
an NU or a DNU propositional default theory and let L be a set of literals. Deciding 
whether A\= Lis 0{'n?), where n is the size of the theory A. 

Based on the above properties, we are now ready to describe the algorithm Outlier 
Enumeration which, for a fixed integer k, enumerates in polynomial time all the strong 
outlier sets of size at most k in an acyclic NU default theory. 

The algorithm is presented in Figure 3. The SCCs Ci, . . . , Cn of the atomic depen- 
dency graph of the theory are ordered such that there do not exist Q and Cj with i < j 
and two letters I e Ci and q e Cj such that there exists a path from letter{q) to letter{j). 

By Theorem 5.8, the cost of steps 7 and 9 is O(n^). Thus, the cost of the algorithm is 
0(2°(n/c) • (cn^ + n^cn^)) = 0{2'^n''~^^). Since c and k are fixed, the algorithm enumer- 
ates the strong outliers in polynomial time in the size of {D, W). For example, all the 
singleton strong outlier sets can be enumerated in time ©(n"). 

6. DISCUSSION AND CONCLUSIONS 

Outlier detection is a knowledge discovery task which has its roots in statistics, ma- 
chine learning, and data mining [Hawkins 1980; Mitchell 1997; Tan et al. 2005]. It 
represents an active research field that has many applications in all those domains 
that can lead to illegal or abnormal behavior, such as fraud detection, network intru- 
sion detection, medical diagnosis, and many others [Hodge and Austin 2004; Chandola 
et al. 2009]. 

Approaches to outlier detection can be classified in supervised, semi-supervised, and 
unsupervised. Supervised methods exploit the availability of a labeled data set, con- 
taining observations already labeled as normal and abnormal, in order to build a model 
of the normal class [Chawla et al. 2004]. Semi-supervised methods assume that only 
normal examples are given. The goal is to find a description of the data, that is a rule 
partitioning the object space into an accepting region, containing the normal objects, 
and a rejecting region, containing all the other objects [Scholkopf et al. 1995]. Unsu- 
pervised methods search for outliers in an unlabelled data set by assigning to each 
object a score which reflects its degree of abnormality [Knorr and Ng 1998; Breunig 
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et al. 2000; Aggarwal and Yu 2001; Papadimitriou et al. 2003; AngiuUi and Pizzuti 
2005; Kriegel et al. 2008; Liu et al. 2008; AngluUi and Fassetti 2009]. 

Traditional approaches model the normal behavior of individuals by performing 
some statistical kind of computation on the given data set and, then, single out those 
individuals whose behavior or characteristics significantly deviate from normal ones. 
However, a very interesting direction of research concerns the capability of exploit- 
ing domain knowledge in order to guide the search for anomalous observations. In- 
deed, while looking over a set of observations to discover outliers, it often happens 
that there is some qualitative description of the domain of interest encoding what an 
expected normal behavior should be. This description might be, for instance, derived 
by an expert and might be formalized by means of a suitable language for knowledge 
representation. 

With this aim, in [AngiuUi et al. 2007; AngiuUi et al. 2008] a notion of outlier in the 
context of some knowledge-based systems has been defined. In particular, the formal 
frameworks considered are those of Default Logics and Extended Disjunctive Logic Pro- 
gramming under answer set semantics [AngiuUi et al. 2008] and Logic Programming 
under stable model semantics [AngiuUi et al. 2007]. 

The outlier detection technique investigated here is based on the definition intro- 
duced in [Angiulli et al. 2008], which is an unsupervised one, in that no examples 
of normality/abnormality are required. This technique can be applied to databases 
including observations to be examined and provided by the organization which is in- 
terested in learning exceptional individuals. In order to exploit domain knowledge, 
databases are to be coupled with a knowledgebase composed of default rules and obser- 
vations. The default rules can be supplied directly by the knowledge engineer or they 
can be a product of a rule learning module. E.g., the rule learning step can be based on 
theory and algorithms developed for learning default rules [Duval and Nicolas 1999] 
and/or for metaquerying [Shen et al. 1996; Ben-Eliyahu-Zohary et al. 2003; Angiulli 
et al. 2003], where metaquerying is a tool for learning rules that involve several rela- 
tions in the database and a metaquery directs the search by providing a pattern of the 
rules of interest. 

Thus, within such a framework, an engine for the outlier detection technique here 
investigated will be able to automatically identify irregular individuals or phenomena 
recorded in the available data and will report on them. 

As a major contribution of this paper, we have analyzed the tractability border as- 
sociated with outlier detection in default logics. Before closing the paper, some further 
comments are in order concerning the technical results presented above. 

From Theorems 4.2 and 4.3, it is clear that neither acyclicity nor strongness alone 
are sufficient to achieve tractability. However, if both constraints are imposed together, 
the complexity of the outlier recognition problem falls within the tractability frontier, 
as shown in Theorem 5.7 reported in Section 5. 

In order to better understand the rationale for the Strong Outlier Recognition prob- 
lem to become tractable for acyclic NU theories, we informally depict why the tech- 
niques exploited in the proofs of Theorems 4.2 and 4.3 fail in this case. By Lemma 
4.1, it follows that, despite the difficulty to encode the conjunction of a set of literals 
through a NU theory, a CNF formula can still be evaluated by means of condition 
1 of Definition 2.2 applied to an acyclic NU theory. In particular, in the construc- 
tion of Lemma 4.1, the letters ci, . . . ,Cm play the role of encoding the truth values 
of the conjuncts Ci, . . . , Cm composing the 3CNF formula $ = f{X), while the sub- 
set S{<^) of X = {xi, . . . ,x„} encodes a truth value assignment to the variables in 
the set X (specifically, is true iff it does not belong to S{^)). Hence, checking for 
(Z)($), W{^)s(<s>)) \= ~'{S{^) U {ci, . . . , Cm}) is equivalent to verifying if Ci A ... A Cm is 
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true under the truth value assignment encoded by S{^) (see Lemma 4.1 for the formal 
proof). 

Now, in the presence of cyclicity, it is possible to constrain the witness set S to contain 
all the letters ci, . . . , c^, by including them in the same SCC of the atomic dependency 
graph and, thus, to resort to Lemma 4.1 in order to prove NP-hardness (the reader is 
referred to Theorem 4.3 for the details). 

In the acyclic case, it is still possible to constrain the witness set S to contain all the 
letters ci, . . . , c^, by exploiting condition 2 of Definition 2.2 as done in the reduction of 
Theorem 4.2. In particular, the reduction introduces a dummy letter cq in the theory 
such that (£), Ws^l) H= ~"^o- As for the letters ci, . . . , Cm, their negation is entailed both 
by {D. Ws) and by {D,Ws,l)- Thus, in order to satisfy condition 2 of Definition 2.2, co 
must be in S. However, in order for {D, Ws) |= -•cq to hold, it must be the case that all 
letters Ci {1 < i < m) are in S. In terms of the atomic dependency graph, this has been 
obtained by adding an arc from each to cq together with an arc from I to cq (recall 
that L = {^1} in the reduction), without the need of introducing cycles in the graph. 

However, if both acyclicity and outlier strongness are imposed, in order to exploit 
Lemma 4.1, it is needed to guarantee that for each Cj, {D, Ws,l) ^ ~'Ci without the 
possibility of introducing cycles in the atomic dependency graph. Informally speaking, 
a possibility would be to add an arc from / to any c^, so that {D,Ws,l) |= Q ( hence 
this theory does not entail -iq), but the graph would become cyclic (see the graph 
in Figure 2). Moreover, removing the dummy letter cq from the reduction in order to 
break the cycle would not help, since in this case each singleton set {q} would act 
as a potential strong outlier witness set for L — {^l}. It can be intuitively concluded 
that requiring acyclicity and strongness simultaneously prevents the size of the min- 
imal outlier witness to be unbounded (this intuition is indeed formalized by Lemma 
5.6) and, hence, prevents the feasibility of reducing satisfiability to the strong outlier 
recognition problem by exploiting the technique depicted in Lemma 4.1. Indeed, if the 
size of S is bounded, CNFs having more than \S\ conjuncts cannot be evaluated by 
means of condition 1 of Definition 2.2. 

To conclude, we note that switching to acyclic theories, general outliers, and NMU 
theories makes the cost of the Outlier Enumeration algorithm exponential. This is 
justified by the results provided in Section 4. Moreover, we point out the following 
valuable observations: 

— For strong outliers, if c is unbounded (on acyclic theories) the cost of the algorithm 
depends exponentially on c (see Theorem 4.3 in Section 4 for the proof of the in- 
tractability of the Strong Outlier Recognition problem on acyclic NU theories). 

— For general outliers, the algorithm still works provided that in line 4 the set S is 
constrained to be a subset of Ci U . . . U Cn, instead of a subset of Ci, and that in line 
6 the set L is constrained to be a subset of Ci U . . . U C,„, where w = max{j : / e 
S and letter{l) e C^}, instead of a subset of Ci U . . . U Cj (note that w > i). However, 
in this case, the algorithm depends exponentially on n, even if it takes advantage 
of the structural property provided by the Incremental Lemma (see Theorem 4.2 in 
Section 4 for the proof of the intractability of the Outlier Recognition problem on NU 
theories). To break the exponential dependency on n, the size of the outlier witness 
set 5* should be constrained to be within a certain fixed threshold h, that is l^j < /i. 

— The algorithm can be applied also to NMU theories, but in this case the cost of steps 
5 and 7 depends exponentially on the size of the theory (the reader is referred to 
Theorem 4.5 in Section 4 for the proof of the co-NP-completeness of the entailment 
problem for propositional (acyclic) NMU theories). 

Overall, the results and arguments reported in this and previous sections indicate that 
outlier recognition, even in its strong version, remains challenging and difficult on 
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default theories. The tractabihty results we have provided nonetheless indicate that 
there are significant cases which can be efficiently implemented. A complete pack- 
age for performing outlier detection in general default theories might therefore try 
to attain reasonable efficiency by recognizing such tractable fragments. Techniques by 
which the outlier detection task in default logics can be rendered practically affordable 
remain a major subject area for future research. 
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